{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.1 Classification\n",
    "\n",
    "* Classification\n",
    "\n",
    "$y\\in \\lbrace 0,1\\rbrace$\n",
    "\n",
    "$0: \"Negative class\"$\n",
    "\n",
    "$1: \"Positive class\"$\n",
    "\n",
    "* It is not recommended to use linear regression for classification problems\n",
    "\n",
    "> For historical reasons, logistic regression is a classification algorithm\n",
    "\n",
    "![](../pic/QQ截图20201023090924.png)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.2 Hypothesis Representation(假设陈述)\n",
    "\n",
    "* Linear Regression Model\n",
    "\n",
    "$$\n",
    "h_{\\theta}(x)=\\theta^Tx\\\\\n",
    "$$\n",
    "\n",
    "> 若 $\\theta^Tx=0$，则说明该$x$处在假设边界上\n",
    "\n",
    "* Logistic Regression Model\n",
    "\n",
    "$$\n",
    "h_{\\theta}(x)=g(\\theta^Tx)\\\\\n",
    "其中，g(z)=\\frac{1}{1+e^{-z}}\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x1b22a4e1128>"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWQAAADzCAYAAACxD2xvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XlYVNfBBvD3wrDvOzoDCo6ggIg6uESz1CQFl2Bi0oZsrSGWqqTZ088k9Uti26+atFmptbbWxMRoYmJCTAW1MSaaJhpcEEFlERSGbUD2bZiZ8/2BoaEaA8hwZ3l/z8Mj4xyHdx6Gl+O5596RhBAgIiL5OcgdgIiIerGQiYgsBAuZiMhCsJCJiCwEC5mIyEKwkImILAQLmYjIQrCQiYgsBAuZiMhCKAY5nqf1kcVLTk5GTk6O3DGIvksayCDOkMnm1NfXyx2BaEhYyEREFoKFTERkIVjIREQWgoVMRGQhWMhERBaChUyySUtLQ3BwMOLi4i57vxACDz30ENRqNeLj43H06NERTkg0sljIJJslS5Zccb9wdnY2iouLUVxcjA0bNmD58uUjmI5o5EmDfAunoZ0Ykr0SqMkf0j8l29bV1YX8k/lI1CRect+ZojPw9fVFSHAIAODQ4UNISEiAi7PLFR9T8/xB5D47xyx5yU6FTgLmrbmaRxjQiSGDPVOPaMTo9Xq4urj23XZxcYG+W3/ZQq6qrkJ1dTUAoKenZ8Qykm0REDCa+n8YTAIX6tsQKQQkaUC9OmQjM0Mm+h7l5eVYuHAhTp48ecl9CxcuxMqVKzFnTu9s98Ybb8TatWuh0Wiu+JgajQa5ublmyUvWw2A04UKHHhfa9Who06O+rbvv88YOPZo6e9Dc0YOmTj2aO3vQ1NGD1i7D9z7eqdXJcHN2HGoczpDJuimVSlRUVPTdrqyshFKplDERWYpugxHVTV2obOyEtqkDNc3dqG3tQm1zV++fLd2ob+vG5eabDhLg5+4MH3cn+Lo5IdjLFeODveDj5gRvNyd4uyrg5aqAl6sTPF3+87mzwvyH3FjIZLFSUlKQmZmJ1NRUHDp0CD4+Phg1apTcsWiEtHUbUKZrx9n6NpTq2nGuoR2VjZ2obOxAXeulZRvg4Yxgb1eEersgbrQPgr1dEeTpjABPF/h7OCPQ0xkBHi7wcXOCg4N5lx6GioVMsrnrrruwf/9+1NfXQ6VS4fnnn+9b/122bBnmz5+PXbt2Qa1Ww93dHZs2bZI5MZlDh96A0zWtOFXdglPVLSipa8NZXTvqWrv7xkgSMNrHDWH+brh2fBBUfm5Q+blD5ecGpa8bgr1d4KIY8nKCxeAaMtkcriFbrtauHpyobMax840oqGrB6ZpWlDe09812vVwUGB/iicggT0QEemBckAcigzwR7u8OVyerLlyuIRORfIQQKNW14ei5JhyraMSx8004U9vaV75jA9wxcZQ3bk1QYuIoL0wc5Q2Vn5vZdzJYMhYyEQ2b6uZOfFnSgC9L6nGwpB66i8sO3q4KTAn3Q3JcKKaG+2FymC983JxkTmt5WMhENGR6gwn/Lq3HvtN1OFhSj7O6dgC9B9iuUQdi9rgAaMb6IzLQw2IPpFkSFjIRDUp7twGfF+mwu6AG+07XobXLADcnR8yI9Mfd08MxWx2I6BAvFvAQsJCJ6Ad19Rixu6AGO/OqcaBYh26DCf4ezpgXF4qk2FDMVgda+0E3i8BCJqLLEkLgeEUTth+pxM68KrR2GTDaxxV3TQ9HclwoNGP8oHDk9cmGEwuZiPqpa+3Ch0e12H6kEiV1bXB1csD8uFG4Q6PCzIgALkWYEQuZiAAAZ2pa8bcDZ/Hx8SrojSZMG+OHNYsnYUH8KHi5ckfESGAhE9kxIQQOltTjbwfK8EWRDm5OjkidHoafXzMW44I85Y5nd1jIRHbIaBLIOq7Fhi/O4nRNK4K8XPBkUjTumREOX3dnuePZLRYykR0RQmB3QS3+uOcMSuraEB3ihRfuiMeihNE2cS0Ia8dCJrITX5U2YG3OaRyvaEJkkAf+cs9UJMeF2vWpypaGhUxk405qm/HC7jP4okiHUT6ueOH2eCyequSWNQvEQiayUc0dPViTcxpbD5+Hr7sTnpk/EffNGsMTOCwYC5nIxggh8HFeFX77SSEaO3qwdE4EHrppPLy5dc3isZCJbEh5fTtWZZ3EgeJ6TA7zxZtpcYgd7SN3LBogFjKRDdAbTNjwRSle21cCF0cHrF4Ui3tmjIEjz6qzKixkIitXVt+Oh7YeQ762GQsmjcL/3hKDEG9XuWPRELCQiazYjqOVWPXRSSgcHbD+3mlIjguVOxJdBRYykRVq7erBqo9O4qPjVZge4Y9X7kzAaF83uWPRVWIhE1mZ4xVNeGjrMVQ2duCxm6OQ8SM114ptBAuZyEoIIfCPL8vxh12nEOLtivd+OQuasf5yx6JhxEImsgJ6gwmrPjqJd3MrkBQbghdunwwfd+4rtjUsZCILd6Fdj2VvH8Hhsgv41Vw1Hr0piheJt1EsZCILVlLXirQ3clHT0oVXUxOwKEEpdyQyIxYykYX6vEiHB7cchYuTI7alz8TUcD+5I5GZsZCJLNBbX5Xj2Y8LEBXihY1LEqHklja7wEImsjCZ+4rxxz1FuGliMF5JnQJPF/6Y2gt+p4kshBACL+4+g3X7S3HbFCVevCOe1yy2MyxkIgtgMgms/qQQb/y7HHfPCMfvFsVxJ4Ud4q9fklVOTg6io6OhVquxZs2aS+5vbm7GLbfcgsmTJyM2NhabNm2SIaV5GU0CT+3Ixxv/LscDcyLw+1tZxvaKhUyyMRqNyMjIQHZ2NgoLC7F161YUFhb2G/PnP/8ZMTExyMvLw/79+/H4449Dr9fLlHj49RhNeOy943g3twIPzVXjNwsm8j3u7BgLmWRz+PBhqNVqREZGwtnZGampqcjKyuo3RpIktLa2QgiBtrY2+Pv7Q6GwjZW2HqMJD75zFFnHq/Dr5Gg89uNolrGdYyGTbLRaLcLCwvpuq1QqaLXafmMefPBBnDp1CqNHj8akSZPw6quvwsHB+l+2JpPAE9vzsLugFs/eEoMVN6jljkQWwPpf2WTTdu/ejYSEBFRVVeH48eN48MEH0dLScsm4DRs2QKPRQKPRQKfTyZB04IQQeG5nQd/M+P7ZEXJHIgvBQibZKJVKVFRU9N2urKyEUtn/1OBNmzZh8eLFkCQJarUaEREROH369CWPlZ6ejtzcXOTm5iIoKMjs2a/Gy3uLsPmrc0i/LhLLrx8ndxyyICxkkk1iYiKKi4tRVlYGvV6Pbdu2ISUlpd+Y8PBwfPrppwCA2tpanDlzBpGRkXLEHRYbD5bhtX0luFMThqfmTeCaMfVjG0dHyCopFApkZmYiKSkJRqMRaWlpiI2Nxfr16wEAy5Ytw6pVq7BkyRJMmjQJQgisXbsWgYGBMicfmvePVOK3nxQiOTYUv78tjmVMl5CEEIMZP6jBRHLQaDTIzc2VO0Y/ewpqsHzLUcyM9Mc/liTCReEodyQaWQP67cslCyIz+6b8Ah7cegxxSh9suE/DMqbvxUImMqOKCx345VtHoPR1wxtLEuHBCwXRFbCQicykrduApW/mwmA04e8/18DPw1nuSGTh+OuayAyMJoGHtx5Dia4Nb9yfiHFBnnJHIivAGTKRGbyQcxqfnq7Ds7fE4Nrxlr0vmiwHC5lomG3PrcBfvziLe2eG42ezxsodh6wIC5loGH1TfgFPf5iP2eoAPHtLrNxxyMqwkImGybc7KlR+7lh39zQ48d0+aJD4iiEaBl09RqzYchQ9F3dU+Lg7yR2JrBB3WRANg9//8xTytc3YcN807qigIeMMmegqfZxXhbe+7r16249jQ+WOQ1aMhUx0FUp1bXjqgxPQjPHDk0nRcschK8dCJhqiTr0RK94+ChcnR7x+9xQexKOrxjVkoiFalXUSRXWtePP+6Rjl4yZ3HLIB/JVONATv5Vbg/SOV+NXc8bguimfi0fBgIRMN0qnqFqz66CRmqwPw8I3j5Y5DNoSFTDQInXojHnznKHzcnPDKnVPg6MB3/aDhwzVkokH4Q/YplOra8fYDMxDk5SJ3HLIxnCETDdBnp+uw+atzWDonAnPGW+f7+pFlYyETDUBDWzeefP8EJoR64QnuNyYz4ZIF0Q8QQuB/PshHS2cP3l46Ha5OfE88Mg/OkIl+wLZvKvCvU7X4dXI0JoR6yx2HbBgLmegKyurbsXpnIeaoA5E2O0LuOGTjWMhE36PHaMIj247BWeGAP/5kMhy4xY3MjGvIRN/j9U+LkVfZjHX3TEWoj6vcccgOcIZMdBl5FU3I/KwEi6cqMX/SKLnjkJ1gIRP9l26DEU++n4dgL1e+Lx6NKC5ZEP2X1z8tQVFtGzYtSYSPG9+KiUYOZ8hE35Ff2Yy/fF6KO6ap8KMJwXLHITvDQia6SG8w4YnteQj0dMaqBTFyxyE7xCULoote31eMM7Wt+McSvms0yYMzZJJVTk4OoqOjoVarsWbNmsuO2b9/PxISEhAbG4vrr7/eLDlOapuxbn8pbp+qwtwJIWb5GkQ/hDNkko3RaERGRgb27t0LlUqFxMREpKSkICbmP8sFTU1NWLFiBXJychAeHo66urphz/HtUkWAhzP+dyGXKkg+nCGTbA4fPgy1Wo3IyEg4OzsjNTUVWVlZ/ca88847WLx4McLDwwEAwcHDf6At87MSnK5pxf/dNolLFSQrFjLJRqvVIiwsrO+2SqWCVqvtN6aoqAiNjY244YYbMG3aNGzevPmyj7VhwwZoNBpoNBrodLoBZyisasG6z0qweIoSN8VwqYLkxSULsmgGgwFHjhzBp59+is7OTsyaNQszZ85EVFRUv3Hp6elIT08HAGg0mgE9ttEksHLHCfi6O2EVlyrIArCQSTZKpRIVFRV9tysrK6FUKvuNUalUCAgIgIeHBzw8PHDdddchLy/vkkIeik1fluFEZTNev2sK/Dycr/rxiK4WlyxINomJiSguLkZZWRn0ej22bduGlJSUfmMWLVqEgwcPwmAwoKOjA4cOHcLEiROv+mtXXOjAn/YUYe6EYCyM57UqyDJwhkyyUSgUyMzMRFJSEoxGI9LS0hAbG4v169cDAJYtW4aJEyciOTkZ8fHxcHBwwNKlSxEXF3dVX1cIgWc+OgkHCfjtrXGQJF5WkyyDJIQYzPhBDSaSg0ajQW5u7vfe/9ExLR559zieuyUGS3jReRoZA/qtzyULsisX2vVY/UkhpoT74r5ZY+WOQ9QPC5nsyu8+KURrVw/W3h4PR74DCFkYFjLZjc+LdNhxTIvl149DVIiX3HGILsFCJrvQoTfgmQ/zMS7IAxlz1XLHIbos7rIgu/DKv4pR2diJ9345Cy4KR7njEF0WZ8hk8wqqmrHxYBnumh6O6RH+csch+l4sZLJpRpPA0zvy4efuhJXJE+SOQ3RFLGSyaW9/fQ55lc1YtTCGV3Iji8dCJptV09yFF3efwXVRQUiZPFruOEQ/iIVMNuu5jwvQYzThd4t4ejRZBxYy2aS9hbXIKajBwzeNR3iAu9xxiAaE297IZrQ3t+PEgVO4UNuM/9l2FOODPPCLayPljkU0YCxksgntze14+7fvo6WhDY09Jpi6jVhQrYW+rRNOPh5yxyMaEC5ZkE04ceAUWhraYAwLQYfkgESFAd4NTThx4JTc0YgGjIVMNqGqtBYuXm7I6nGGA4CbFHq4ebmhurRW7mhEA8ZCJpswelwIDhudUC0c4SOZ4CYBna2dGDWOb1xK1oOFTDYhaHIEDnn6YUx3B5yNRtSe08E7wBPx11792z0RjRQWMtmEtZ+VwdHJEY/PDoezixPm3D4D9666Ax48oEdWhIVMVi/nZA32Ftbi0ZujkHLHDPgG+2DWQg3LmKwOC5msWmtXD577uAATQr2QNofvj0fWjfuQyar9aU8Ralu78Jd7p8LJkfMLsm58BZPVyqtowptfleO+mWMwJdxP7jhEV42FTFbJYDThqR35CPJ0wRNJ0XLHIRoWXLIgq7Tpy3IUVrdg3T1T4e3K6xyTbeAMmaxOxYUOvLS3CDdOCMa8uFC54xANGxYyWRUhBJ7+MB8OErD6Vl7nmGwLC5msykfHtThQXI8nk6Kh9HWTOw7RsGIhk9VoaOvG6p2FmBLui/tmjZU7DtGwYyGT1fjdP0+hrduAtbfHw9GBSxVke1jIZBU+L9Lhw2NaLL9+HKJCvOSOQ2QWLGSyeB16A575MB/jgjyQMVctdxwis2Ehk6xycnIQHR0NtVqNNWvWXHbMS3uKUNnYia9e+xV2fvThCCckGjksZJKN0WhERkYGsrOzUVhYiK1bt6KwsLDfmLyKJvzjyzJ41R7HDbFhMiUlGhksZJLN4cOHoVarERkZCWdnZ6SmpiIrK6vv/h6jCSt35MNNMuD+qX4IDg6WMS2R+bGQSTZarRZhYf+Z9apUKmi12r7b6z4rxanqFrgXfoxHMpZd8bE2bNgAjUYDjUYDnU5ntsxE5sRCJotUWNWC1/cVw7+1FK+v/AUcHK78Uk1PT0dubi5yc3MRFBQ0QimJhhcvLkSyUSqVqKio6LtdWVkJpVKJHqMJT76fB193Z9R/sAGp/3wJAFBfX49du3ZBoVDg1ltvlSs2kdmwkEk2iYmJKC4uRllZGZRKJbZt24Z33nkHf9lfioKqFvz1vmlI+s3JvvFLlizBwoULWcZks1jIJBuFQoHMzEwkJSXBaDQiLS0NDv5heOWtLxDj0YGkWF7JjeyLJIQYzPhBDSYajB6jCbet+xI1zV3Y8+j18PdwHtLjaDQa5ObmDnM6oqsyoHP9OUMmi/HXz0txUtuC9fdOHXIZE1kz7rIgi3CmphWvflqMhfGjkBw3Su44RLJgIZPsDEYTntieB29XJzyfEit3HCLZcMmCZPf6vhLka5ux7p6pCPB0kTsOkWw4QyZZHTnXiNf3FWPxFCXmT+JSBdk3FjLJpq3bgEffPY7Rvm54fhGXKoi4ZEGyWb2zAJWNHdiWPgterk5yxyGSHWfIJIuck9V4L7cSy28Yh+kR/nLHIbIILGQacbUtXVi5Ix/xKh88clOU3HGILAYLmUaUySTwxPY8dPUY8fKdCXBy5EuQ6Fv8aaAR9eZX5ThQXI/fLIjBuCBPueMQWRQWMo2YMzWt+EP2adw4IRj3zAiXOw6RxWEh04ho7zZg+ZYj8HZ1wprb4yFJA7rWCpFdYSGT2Qkh8PSH+Sivb8drdyUgyItn4xFdDguZzG7LofPIOl6Fx26OwjXjAuWOQ2SxWMhkVie1zVi9sxDXRwVhxQ1queMQWTQWMplNc2cPlm85ggBPZ7x8ZwIcHLhuTHQlPHWazEIIgSe356G6qQvv/nIWLzhPNACcIZNZbDxYhj2FtVg5bwKmjfGTOw6RVWAh07A7cu4C1mSfRlJsCB6YEyF3HCKrwUKmYVXV1Illbx+F0s8NL9wxmfuNiQaBa8g0bDr0Bix9MxddeiPeWToDPm68pCbRYLCQaViYTAKPvZuH0zUt2LgkEeNDvOSORGR1uGRBw+LlfxUhp6AGT8+fiB9FB8sdh8gqsZDpqmUd1+L1fSVITQzjQTyiq8BCpqty7Hwjnnz/BGZE+GP1ojgexCO6CixkGrKqpk6kv3UEod6u+Mu90+Cs4MuJ6GrwoB4NSXNnDx54MxedeiO2LJ3BM/GIhgGnNDRonXojlr75DUrqWrHunqmI4o4KomHBGTINSo/RhBVbjiD3XCNev2sKrosKkjsSkc3gDJkGzGQSePy9PHx2Roff3zoJC+NHyx2JyKawkGlAhBB49uMCfJxXhV8nR+PuYXpPvJycHERHR0OtVmPNmjWX3L9lyxbEx8dj0qRJuOaaa5CXlzcsX5fIEnHJggbk5b1FeOvrc0i/LhLLrx83LI9pNBqRkZGBvXv3QqVSITExESkpKYiJiekbExERgc8//xx+fn7Izs5Geno6Dh06NCxfn8jScIZMP2jjwTK8tq8EP9Wo8NS8CcO21/jw4cNQq9WIjIyEs7MzUlNTkZWV1W/MNddcAz+/3st3zpw5E5WVlcPytYksEQuZrmjjwTL89pNCJMWG4P9umzSsJ35otVqEhYX13VapVNBqtd+fZeNGzJs377L3bdiwARqNBhqNBjqdbtgyEo0kLlnQ98rcV4w/7inCvLhQvJo6BQpH+X5/f/bZZ9i4cSMOHjx42fvT09ORnp4OANBoNCMZjWjYsJDpEkIIvLj7DNbtL8VtU5R48Y54s5SxUqlERUVF3+3KykoolcpLxp04cQJLly5FdnY2AgIChj0HkaXgkgX1YzIJPL+zEOv2l+LuGeH4008mm21mnJiYiOLiYpSVlUGv12Pbtm1ISUnpN+b8+fNYvHgx3nrrLURFRZklB5Gl4AyZ+hhNAk/vyMe7uRV4YE4EfrNgolkvFqRQKJCZmYmkpCQYjUakpaUhNjYW69evBwAsW7YMq1evRkNDA1asWNH3b3Jzc82WiUhOkhBiMOMHNZisR4/RhMffy8PHeVV4aK4aj94cZbVXbtNoNCxtsjQD+mHiDJnQ2K7H8i1H8PXZC/h1cjRW3KCWOxKRXWIh27mSujYsffMbVDV14aWfTsbiqSq5IxHZLRayHTtQrMOKLUfh7OiArekzMG2Mv9yRiOwaC9lOvfVVOZ7bWQh1kCf+/nMNwvzd5Y5EZPdYyHbGYDRh9SeF2PzVOdw4IRiv3jUFni58GRBZAv4k2pHKxg48su04cs814hfXRmDlvIlwdLDOnRREtoiFbCd25Vdj5QcnYBLAK3cm4NYpl54RR0TyYiHbuE69Eas/KcDWwxWYHOaL11ITMCbAQ+5YRHQZLGQbdqq6Bb/aegylujYsv2EcHrs5Ck4yXiCIiK6MhWyDDEYT3vh3OV7YfQY+bk54K20G5owPlDsWEf0AFrKNOVHZhKd25KOgqgU3TQzG2tvjEeDpIncsIhoAFrKNaO3qwZ/2FGHzV+UI9HTBunumYl5cqNVej4LIHrGQrZwQAjkna/DczgLUtXbjZzPH4PGkaHi7OskdjYgGiYVsxU5Vt2BtzmnsP6NDzChv/PU+DRLCfOWORURDxEK2QucbOvDS3jPIyquCl4sCv1kwEUuuGSvrWywR0dVjIVuRutYuZO4rwdbD5+HoIGHZ9eOw7Lpx8HHn8gSRLWAhW4H6tm5s+rIM/zhYDr3RhNTEMDx043iEeLvKHY2IhhEL2YKV1LXi7wfKsOOYFnqDCbdMHo3Hbo5CRCDPtCOyRSxkCyOEwFelDfjbgbP47IwOLgoH3DFNhQfmRGBckKfc8YjIjFjIFqKlqwc786rwzqHzKKhqQYCHMx69KQr3zgzniR1EdoKFLCOTSeDfpQ3YfqQCOSdr0G0wITrEC39YPAm3TVHC1clR7ohENIJYyDIoqWvFx8er8MFRLbRNnfB2VeCnmjD8RKPCJKUPz64jslMs5BFgMgmc0DZjd0ENdhfU4KyuHZIEXDs+CCvnTcDNMSGcDRMRC9lcOvVGHC6/gE9P1WJPQS1qWrrg6CBhZqQ/7r9mLH4cG8pta0TUDwt5mBiMJuRrm/FlST0OltTj6Lkm6I0muDo54LrxQXgyNho3TgyGr7uz3FGJyEKxkIeovduAvMomHDvfhGPnG3Ho7AW0dhsAALGjvbFk9ljMVgdi+lh/uDlzOYKIfhgLeQC6DUYU17ahsLoFxyt6S/hMTQtMovf+yCAPLJw8CrPVgZgVGcBtakQ0JCzk7zCZBKqaO1Gqa8ep6pa+j1JdO4wX29fLVYGEMF/cPHc8poT7YkqYL5chiGhY2F0hm0wCurZuVDZ24FxDB87q2nG2vg1nde0ob2hHV4+pb+xoH1dMGOWNm2NCMCHUGxNHeSMy0AMODtyWRkTDz6YKWQiBpo4e1LZ2oaa5C3Ut3aht6UJVcycqG3s/tI2d0Bv/U7qODhLC/d0REeiBOepARAZ5IjLIAxNCvTjzJaIRZdGF3GM0obmzB00dPWju1KOpo/fzC+161Ld3o6FNjwvtejS0daO+TQ9da3e/sv1WoKczVH7uiBntjR/HhkDl5w6VnxvC/NwR7u8OZwWvI0xE8huRQv6m/AIutOvRoTegvdvY78+2bgNaugxo6zKgtasHrV0X/66zB+164/c+prOjAwI8nRHg6Qx/DxdEBnki2NsFIV6uCPVxRYi3C4K9XBHs7QIXBXc5WKqcnBw8/PDDMBqNWLp0KVauXNnvfiEEHn74YezatQvu7u544403MHXqVJnSEpnXiBTyUzvyUVLXdsnfuzk5wsNFAW9XBbxcFfB0VSDYyxVergp4uTrB1733w8fNCb7uzvB16/08wNMZni4KnmJs5YxGIzIyMrB3716oVCokJiYiJSUFMTExfWOys7NRXFyM4uJiHDp0CMuXL8ehQ4dkTE1kPiNSyK/cmQAA8HBRwMPZEe4uCrg5OcKRB8fs2uHDh6FWqxEZGQkASE1NRVZWVr9CzsrKws9+9jNIkoSZM2eiqakJ1dXVGDVqlFyxicxGEkIMeHBycrKor683Yxzz0Ol0CAoKkjvGiLP0593Y2IiWlhaMGTMGANDQ0ID29naEh4f3jSkpKUFoaCg8PXuvBV1UVASVSgV3d/d+j6XT6fDta7O7uxsJCQkj9Cwsh6V/v83FGp73kSNHdgshkn9woBBiMB9Wadq0aXJHkIWlP+/t27eLBx54oO/25s2bRUZGRr8xCxYsEAcOHOi7PXfuXPHNN99c8XHd3d2HN6iVsPTvt7lYyfMeUMdyewHJRqlUoqKiou92ZWUllErloMcQ2QoWMskmMTERxcXFKCsrg16vx7Zt25CSktJvTEpKCjZv3gwhBL7++mv4+Phw/ZhslkXvQx4u6enpckeQhaU/b4VCgczMTCQlJcFoNCItLQ2xsbFYv349AGDZsmWYP38+du3aBbVaDXd3d2zatOkHHzcwMNDc0S2SpX+/zcWWnvegDuoBGNRgIjloNBrk5ubKHYPouwa0pYxLFkREFoKFTERkIQa7ZGH1JEl6HMAfAQQJIaxvU/UgSZL0IoBbAOgBlAK4XwjRJG8q85EkKRnABwCqAfxdCLFG5khmJ0lSGIDNAELQu6y4QQjxqrwleorRAAACJUlEQVSpRo4kSY4AcgFohRAL5c5zNexqhnzxhftjAOflzjKC9gKIE0LEAygC8JTMeczm4g/mnwFMAhAD4C5JkmKu/K9sggHA40KIGAAzAWTYyfP+1sMATskdYjjYVSEDeBnAr2FHByeFEHuEEIaLN78GoJIzj5lNB1AihDgrhNAD2AZgkcyZzE4IUS2EOHrx81b0lpNdbNaWJEkFYAGAv8udZTjYTSFLkrQIvf+lyZM7i4zSAGTLHcKMlAAqvnO7EnZSTN+SJGksgCkA7OUKTK+gd5J16XV3rZBN7UOWJOlfAEIvc9czAJ5G73KFzbnS8xZCZF0c8wx6/2u7ZSSz0ciRJMkTvevnjwghWuTOY26SJC0EUCeEOCJJ0g1y5xkONlXIQoibLvf3kiRNAhABIO/iJTtVAI5KkjRdCFEzghHN4vue97ckSVoCYCGAG4VtH8XVAgj7zm3Vxb+zeZIkOaG3jLcIIXbInWeEzAaQIknSfACuALwlSXpbCHGvzLmGzO52WQCAJEnlADR2sssiGcBLAK4XQujkzmNOkiQp0Hvg8kb0FvE3AO4WQhTIGszMpN5ZxpsALgghHpE7jxwuzpCf4C4LsnSZALwA7JUk6bgkSevlDmQuFw9ePghgN3oPbL1n62V80WwA9wGYe/F7fPzirJGsjF3OkImILBFnyEREFoKFTERkIVjIREQWgoVMRGQhWMhERBaChUxEZCFYyEREFuL/AWlXX6ONzQCcAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1b22a3ea048>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Draw sigmoid function graph\n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "def sigmoid(x):\n",
    "    return 1 / (1 + np.exp(-x))\n",
    "\n",
    "def y1(x):\n",
    "    return x/x\n",
    "\n",
    "fig = plt.figure()\n",
    "\n",
    "x = np.arange(-5, 5, 0.2)\n",
    "y1 = sigmoid(x)\n",
    "y2 = [1 for item in range(50)]\n",
    "\n",
    "N = 50\n",
    "area = np.pi * (15 * np.random.rand(N))**2  # 0 to 15 point radii\n",
    "ax = fig.add_subplot(1, 1, 1)\n",
    "\n",
    "# 设置轴的位置\n",
    "ax = plt.gca()  # get current axis 获得坐标轴对象\n",
    "ax.spines['right'].set_color('none')  # 将右边 边沿线颜色设置为空 其实就相当于抹掉这条边\n",
    "ax.spines['top'].set_color('none')\n",
    "ax.xaxis.set_ticks_position('bottom')\n",
    "ax.yaxis.set_ticks_position('left')\n",
    "ax.spines['bottom'].set_position(('data', 0))  # 指定 data 设置的bottom(也就是指定的x轴)绑定到y轴的0这个点上\n",
    "ax.spines['left'].set_position(('data', 0))\n",
    "# plt.xticks(rotation=45)#x轴数值倾斜45度显示\n",
    "\n",
    "plt.plot(x, y1)\n",
    "plt.plot(x, y2)\n",
    "plt.scatter(0, 0.5, s=30, c=1, alpha=0.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.3 Decision boundary(假设边界)\n",
    "\n",
    "* logistic regression\n",
    "\n",
    "$$\n",
    "h_{\\theta}(x)=g(\\theta^Tx)=P(y=1|x;\\theta)\n",
    "$$\n",
    "\n",
    "* predict: \"$y=1$\" if $h_{\\theta}(x)\\geq 0.5$\n",
    "\n",
    "* predict: \"$y=0$\" if $h_{\\theta}(x)< 0.5$\n",
    "\n",
    "![](../pic/QQ截图20201023151901.png)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.4 Cost function\n",
    "\n",
    "* Linear regression: \n",
    "\n",
    "$$\n",
    "J(\\theta_{0}, \\theta_{1})=\\frac{1}{2m}\\sum_{i=1}^{m}(h_{\\theta}(x^{(i)})-y^{(i)})^2\\\\\n",
    "\\quad =\\frac{1}{m}\\sum_{i=1}^{m}\\frac{1}{2}(h_{\\theta}(x^{(i)})-y^{(i)})^2\\\\\n",
    "\\quad =\\frac{1}{m}\\sum_{i=1}^{m} cost(h_{\\theta}(x^{(i)}),y)\\\\\n",
    "cost(h_{\\theta}(x),y) = \\frac{1}{2}(h_{\\theta}(x)-y)^2\n",
    "$$\n",
    "\n",
    "* Use the loss function of linear regression to logistic regression: \"non-convex\"\n",
    "\n",
    "![](../pic/QQ截图20201023153601.png)\n",
    "\n",
    "* the cost of logistic regression\n",
    "* make sure: \"convex\"\n",
    "\n",
    "$$\n",
    "Cost(h_{\\theta}(x^{(i)},y^{(i)}))\\begin{cases} -log(h_{\\theta}(x), & \\text {if $y$ = 1}\\\\ -log(1-h_{\\theta}(x), & \\text{if $y$ =0} \\end{cases}\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.5 Simplified cost function and gradient descent\n",
    "\n",
    "* logistic regression cost function\n",
    "\n",
    "$$\n",
    "J(\\theta_{0}, \\theta_{1})=\\frac{1}{m}\\sum_{i=1}^{m} Cost(h_{\\theta}(x^{(i)}),y^{(i)}) \\quad (y=0,1)\\\\\n",
    "Cost(h_{\\theta}(x^{(i)},y^{(i)}))\\begin{cases} -log(h_{\\theta}(x), & \\text {if $y$ = 1}\\\\ -log(1-h_{\\theta}(x), & \\text{if $y$ =0} \\end{cases}\\\\\n",
    "$$\n",
    "\n",
    "* define:\n",
    "\n",
    "$$\n",
    "Cost(h_{\\theta}(x),y)=-y \\log (h_{\\theta}(x)-(1-y) \\log(1-h_{\\theta}(x))\\\\\n",
    "J(\\theta)=\\frac{1}{m}\\sum_{i=1}^{m}Cost(h_{\\theta}(x^{(i)}),y^{(i)})\\\\\n",
    "=-\\frac{1}{m}[ \\sum_{i=1}^{m}y^{(i)} \\log (h_{\\theta}(x^{(i)})+(1-y^{(i)}) \\log(1-h_{\\theta}(x^{(i)}))]\\\\\n",
    "$$\n",
    "\n",
    "* 求导得：\n",
    "\n",
    "![](../pic/QQ截图20201023175815.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.6 Advanced optimization(高级优化)\n",
    "\n",
    "* Optimization algorithms:\n",
    "   * Greadient descent\n",
    "   * Conjugate gradient\n",
    "   * BFGS\n",
    "   * L-BFGS\n",
    "   \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.7 Multi-class classification\n",
    "\n",
    "![](../pic/QQ截图20201023193436.png)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
